Best Practices Overview

Overview

This is an introduction to key Best Practices as applied to Algonomy’s Recommend. Following the guidelines helps safeguard performance of Recommend on a retailer's site. Following the guidelines set forth in the pages that follow will help safeguard performance of Recommend on a retailer's site.

All sites are different, and what works best for one is not guaranteed to translate into gains for another. As such, treat this guidance as general best practices, a foundation for optimization from which site-specific tuning should occur. Understand the logic behind our optimization methodology, the questions to ask, so that you can adapt and apply as appropriate.

Best Practices

The intent of these best practices is to navigate you through common user experience decisions when setting up Recommend for a retail site. In addition to providing specific guidance on the baseline best practice recommendations configuration, the objective is to help you to understand the logic behind our decision making, so that you’re prepared to make informed adjustments specific to a retail site.

After reading the documents within the guide, you can answer the following questions:

How do we measure the effectiveness of recommendations?
On what page types are recommendations most important?
How many placements should we use and where should we position them?
What should recommendations look like?
What strategies should we use for each page type?
What are the user experience caveats and considerations that we should be mindful of when configuring a site with recommendations?
Which special considerations should retailers keep in mind when configuring recommendations in mobile experiences?

Measuring Recommendations

Value Measurement

Assessing the quantitative value of recommendations requires that you run an A/B test which compares the Revenue Per Visitor (RPV), Conversion (CVR), and Average Order Value (AOV) of customers who are exposed to product recommendations with those customers who are not. Typically, RPV is the primary success metric, though some retailers may prioritize CVR due to the high lifetime value (LTV) of their customer base; that is, they’re willing to sacrifice basket size to get a sale of any value, because once they gain a customer, they’re able to monetize him/her over a long period of time (for example, via lifecycle email marketing campaigns). Therefore, you oftentimes see retailers offer 10-30% off promo codes for new customers.

With Recommend, we expect to deliver 2-5% RPV lift against no recommendations and 1-3% RPV lift against an alternative technology—whether that be a competing recommendations vendor, an internal solution, or manual merchandising. There is also a notion of qualitative value measurement which is a more subjective validation of whether the experience is delightful, compelling, and useful to customers. These evaluations typically require user testing (focus groups, session monitoring, etc.) and can sometimes confirm improvements that AB testing is not able to validate (i.e., because the lift is not detectable with a high degree of confidence). While we must be mindful of heuristics and strive to create memorable shopping experiences (in fact, we must build these qualitative considerations into our best practices), Algonomy does not have a structured, formal process for such evaluations.

Utilization Measurement

It’s very tempting to use recommendations sales (i.e., attributables), that is revenue from items clicked on recommendations, as a proxy for solution value. At Algonomy, we consider recommendations sales as a utilization or engagement metric but not as a good indicator of value. Recommendation sales measure the recommendation consumption and relevance to customers; customers will not engage with recommendations if they do not find the content germane to their shopping experience. However, the recommendation sales metric does not internalize the following phenomena, each of which contributes to solution value:

Halo Effect: Sales of incremental non-recommended items due to the presence of recommendations. Recommendations create stickiness and elicit deeper shopper engagement with your product catalog. By keeping customers onsite, recommendations increase the opportunity for customers to purchase non recommended products.
Customer Distraction: Conversion loss due to the presence of recommendations. If improperly utilized, recommendations can cause customers to over-explore and not convert. An extreme example would be a site with 5 prominently positioned recommendations placements on each page. This scenario would lead to a lot of clicks followed by customer fatigue, and ultimately, a non-conversion. Remember, it’s not about maximizing engagement with recommendations, but about achieving the right amount of engagement that maximizes incremental revenue.
Cannibalization: Average Order Value (AOV) loss due to the presence of recommendations. If not configured properly, recommendations can cause customers to buy lower-priced products than they would otherwise purchase. As such, particularly deep in the funnel, it is critical to carefully align recommendations with merchandising objectives.
Overlap: Non-incremental sales that would have happened in the absence of recommendations. In many cases (if not most), recommendations purely accelerate product discovery rather than resulting in a new sale.

By leveraging best practices adapted for your site, you can minimize phenomena 2 through 4 and optimize recommendations to deliver material Revenue Per Session (RPV) lift.

Recommendations User Profiling

Retailers sometimes want to know what kinds of customers utilize recommendations so that they can better understand how we’re delivering incremental revenue; whether we’re catering to their loyal customers or helping them monetize their low-value customers (i.e., the ones at the greatest risk of non-conversion). A cohort analysis provides this insight by reporting the KPI differences, typically RPV, CVR, and AOV, between recs users and non-recs users.

Oftentimes, a cohort analysis will reveal substantially higher spend among users of recommendations. This is usually the case on B2C sites, where the “best” customers tend to engage with recommendations. It’s often the inverse in B2B retail; the highest-spending customers arrive to the site with very specific purchase lists from which they cannot deviate (one could get fired for buying unapproved items). In this case, recommendations cater to the non-committal buyers—organizations with potentially smaller budgets that don’t know exactly what they want, are open to suggestion, and demonstrate high conversion risk. It’s important not to mistake cohort data for a value analysis as it has an obvious, inherent self-selection bias. That is, the bulk of the observed delta between recommendations and non-recommendations users are not due to the value recommendations deliver; it’s due to what types of customers decide to use recommendations.

Guiding Principles

The following precepts guide the formulation of our optimization best practices:

Optimizations must have quantitative and qualitative merit. While our practices are designed to maximize incremental revenue, they must also provide a resonant and relevant shopping experience that delights users and aligns with the retailer’s merchandising and brand objectives. For example, it’s possible we could produce more incremental revenue for a high-end, luxury retailer if we boosted on-sale items across the site; that said, persistent sales promotion is probably inconsistent with their brand, so this is not a viable tactic.
Page objectives must lead us to action. Retailers strive to reduce friction in the shopping funnel and create an elegant flow that moves customers from the Home page through checkout. To do this, each page is assigned a distinct objective that helps transition customers from consideration to commitment. It is critical that our recommendations support these objectives.
Recommendations are not the most important piece of content on the site. Recommendations enhance the shopping experience; they do not define the shopping experience. Over-saturation puts conversion and basket sizes at risk. As such, we must accept that, in many scenarios, recommendations must take a backseat to other forms of content on the page.